Constrained Local Latent Variable Discovery

For many applications, the observed data may be incomplete and there often exist variables that are unobserved but play an important role in capturing the underlying relationships. In this work, we propose a method to identify local latent variables and to determine their structural relations with the observed variables. We formulate the local latent variable discovery as discovering the Markov Blanket (MB) of a target variable. To efficiently search the latent variable space, we exploit MB topology to divide the latent space into different subspaces. Within each subspace, we employ a constrained structure expectation-maximization algorithm to greedily learn the MB with latent variables. We evaluate the performance of our method on synthetic data to demonstrate its effectiveness in identifying the correct latent variables. We further apply our algorithm to feature discovery and selection problem, and show that the latent variables learned through the proposed method can improve the classification accuracy in benchmark feature selection and discovery datasets.